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Abstract 

Undergraduate mathematics programs must prepare teachers for the challenges of 
teaching statistical thinking as advocated in standards documents and statistics 
education literature. This study investigates the statistical thinking of pre-service 
secondary mathematics teachers at the end of their undergraduate educations. 

Although cdl had completed a required upper-division two-course sequence in 
probability and statistics, most were challenged by two tasks which required a 
critical ancdysis of the use of statistics in newspaper articles. Some patterns emerged 
in the incorrect answers, including a tendency to focus on potential sampling issues 
which were not relevant to the tasks. The results have implications for, and reaffirm 
concerns about, the undergraduate statistics preparation of secondary teachers in the 
United States. 

Keywords: pre-service teachers, teacher knowledge, mathematical knowledge for teaching 
(MKT), statistical thinking, statistical knowledge for teaching 

Introduction 

Statistical literacy must be a goal of K-12 education (Franklin et al., 2007); it is essential 
to informed citizenry, to decision-making, and to economic empowerment (Utts, 2003). The 
achievement of this goal is dependent upon the preparation of mathematics teachers who are 
proficient in statistical thinking and can foster that ability in their students. That is, K-12 
teachers must understand and be able to communicate “the need for data, the importance of 
data production, the omnipresence of variability, and the quantification and explanation of 
variability” (Aliaga et al., 2005, p. 14). Indeed, recent United States K-12 standards 
documents, namely, the American Statistical Association’s GAISE standards (Franklin et al., 
2007) and the widely-adopted Common Core State Standards for Mathematics (National 
Governors Association Center for Best Practices & Council of Chief State School Officers, 
2010), have highlighted the importance and need for students to develop statistical literacy 
and engage in statistical thinking. Yet, many in- and pre-service teachers are products of a 
system where the learning of data and chance at the K-12 level has been underemphasized 
(Shaughnessy, 2006). Against this backdrop of increasing statistical demands on K-12 
teachers, this paper reports on research about the statistical thinking of pre-service secondary 
mathematics teachers (PSMTs) at the end of their undergraduate mathematics programs at a 
large university in the western United States. 

In particular, two statistical thinking questions were asked of 22 senior-level students 
enrolled in a capstone mathematics course for undergraduate mathematics majors intending 
to be secondary mathematics teachers. All of the students had completed, as a pre-requisite 
for the course, an upper-division two-semester-long calculus-based probability and statistics 
course sequence offered in a department of mathematics & statistics. Each of the two 
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questions required students to comment on whether statistics reported in a newspaper article 
supported a claim from the article. The students’ performances indicated that many of these 
PSMTs may not have the statistical thinking dispositions or skills necessary to make sense of 
quantitative information reported in the media. Herein, the nature of some of their challenges 
and the implications for secondary teacher preparation are discussed. 

Perspective 

In 2001, the Conference Board of the Mathematical Sciences (CBMS) in the United 
States published a set of recommendations called the Mathematics Education of Teachers 
(2001). They recommended that PSMTs “should have experience formulating questions, 
devising data collection protocols, and analyzing real data sets that result from their own 
investigations or from the data collection of others” (p. 136). Since that time, the Common 
Core State Standards for Mathematics (CCSSM) have been adopted by over 40 states. The 
CCSSM high school standards include, as one of six conceptual categories, Statistics & 
Probability and emphasize interpreting of data, making inferences, and justifying conclusions 
(National Governors Association Center for Best Practices & Council of Chief State School 
Officers, 2010). Furthermore, the CCSSM endorse eight practice standards for all of K-12; of 
particular relevance is the practice of “construct[ing] viable arguments and [critiquing] the 
reasoning of others” (p. 6). In 2012, the CBMS released an updated version of their 
recommendations for teacher preparation which calls for focus “on data collection, analysis, 
and interpretation needed to teach the statistics outlined in the [CCSSM]” (2012, p. 18). This 
type of preparation may not be the status quo; Rossman, Chance, Medina, and Obispo (2006) 
pointed out that many mathematics teachers “do not have ample opportunities to develop 
their own statistical skills and understanding of statistical concepts before teaching them to 
students” (p. 332). They connect this issue to the structure of teacher preparation programs in 
which PSMTs receive little instruction in “communication skills and statistical judgment” (p. 
332). 

Indeed, concerns have been raised within the statistics education community about the 
state of teacher preparation. Some have cautioned that K-12 statistics teachers are not 
sufficiently aware of the differences between mathematical thinking and statistical thinking 
(Burrill & Biehler, 2011; M. Pfannkuch, 2008). Pfannkuch (2008) called for preservice 
teachers to have more authentic statistical experiences, that is, to “[learn] the game of 
statistics.” Burrill and Biehler (2011) called for “direct intervention” in teacher training in 
order for PSMTs to develop the necessary “philosophy of statistics” (p. 66). They note that 
there are several important, unanswered questions in the statistics education community about 
how to best train PSMTs in statistics. Similar concerns and uncertainty have been expressed 
about the preparation of teachers of introductory college statistics courses (Pearl et al., 2012). 

Despite these concerns, there have been few studies of teachers’ statistical knowledge for 
teaching. Groth (2007, 2013) proposed a hypothetical framework for this knowledge and 
lamented that “there is a daunting amount of work to accomplish in building programs that 
are effective in helping teachers develop knowledge for teaching statistics” (2007, p. 433). 
Callingham and Watson (2011) and Watson, Callingham, and Nathan (2009) reported on 
ongoing work to enumerate components of and to develop a measure of teachers’ statistical 
pedagogical content knowledge (PCK). This work, and that of Groth (2007), is built upon the 
notion of PCK, introduced by Shulman (1986), as the subject matter knowledge which is 
relevant to teachers. In particular, teachers of statistics need types of statistical knowledge 
which are specific to teaching; for example, they must be able to choose appropriate 
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representations and anticipate student errors. In particular, they must be able to support the 
statistical thinking expected of students as defined by the CCSSM. 

There are, however, some small scale studies of in- and pre-service teachers’ statistical 
knowledge (e.g., Doerr & Jacob, 2011; Makar & Confrey, 2005a; Maxine Pfannkuch, 2006), 
which indicate that teachers have the same statistical challenges as students, particularly with 
conceptual understanding. Of relevance, a study by Makar and Confrey (2005b) found that 
the 17 PSMTs in their study often used non-standard terminology to discuss statistical 
variation and distribution. However, such language was often used to discuss rich statistical 
ideas and presented opportunity for insight into student thinking and statistical sense-making. 
More recently, the 18th ICMI Study included reviews of the limited research on teachers’ 
graphical knowledge (Gonzalez, Espinel, & Ainley, 2011) and on their understanding of 
averages (Jacobbe & Carvalho, 2011), variation (Sanchez, da Silva, & Coutinho, 2011), and 
distribution (Chris Reading & Canada, 2011). These reports each acknowledged that teachers 
need deep understandings of these topics, yet often encounter difficulties or are deficient in 
this respect. Among these reviews, there are calls for improving teacher training and 
continued research into teacher learning and knowledge. 

Also of relevance to the present study are the notions of statistical knowledge and 
thinking. Utts (2003) connected statistical knowledge to requirements for quantitative literacy 
and for good citizenry. As Garfield and Ben-Zvi (2005) suggested, being able to “provide 
good evidence-based arguments and critically evaluate data-based claims are important skills 
that all citizens should have” (p. 355). Chance (2002) catalogued several definitions for 
statistical thinking. Despite the variety of definitions, it is widely accepted that (1) statistical 
thinking includes an appreciation for variability and the statistical process (in contrast with 
the following of procedures) and (2) statistical thinking is a goal of statistics education. 
Within the constellation of dispositions which comprise statistical thinking, some researchers 
have focused on distributional thinking, the ability to reason by coordinating multiple aspects 
of a distribution and, if applicable, considerations of sample and population (A. Bakker & 
Gravemeijer, 2005; C. Reading & Reid, 2006; Shaughnessy, 2007) 

Chance (2002) noted that “the statistical thinker is able to move beyond what is taught in 
[a statistics] course, to spontaneously question and investigate the issues and data involved in 
a specific context.” She also endorsed explicit instruction in the statistical thinking habits 
which are endemic to professional statisticians. Among the six habits/skills that she lists, and 
of relevance to the present study, are skepticism about data, constant attention to context, and 
“thinking beyond the textbook.” Others have provided tasks or suggestions to promote 
statistical thinking (e.g., Burrill & Elliott, 2006) or distributional thinking (e.g., Arthur 
Bakker, 2004; Makar & Confrey, 2003); Groth (2013) provides examples of activities used 
with pre-service teachers for the specific purpose of developing their SKT. 

Methodology 

The data are comprised of student work on two tasks from a capstone mathematics course 
for PSMTs which was taught by the author of this paper. Question 1 was a homework 
question; there is no information about the level to which students collaborated on this task. 
Question 2 was assigned on a take-home exam; collaboration was prohibited on the exam. 
Both questions required students to examine newspaper quotes. Of the semester-long course, 
three weeks were devoted to discussion of statistics, though no in-class activities 
demonstrated or required students to engage in the interpretation of news reports, though 
Question 1 was discussed by the whole class after it was graded. The choice to limit 
discussions of and instruction in these sorts of analyses was made by the instructor in order to 
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test the hypothesis that the students’ previous coursework did not prepare them for the type of 
statistical thinking required by the two questions. 

Twenty-two students participated in the study; all were enrolled in a semester-long 
mathematics content capstone course for pre-service secondary teachers. The students had 
completed a two-semester-long upper division calculus-based probability and statistics course 
sequence as well as all of the non-elective mathematics courses required for all mathematics 
majors. The sequence was taught from the textbook Mathematical Statistics with 
Applications (Wackerly, Mendenhall, & Scheaffer, 2007) and, though not all instructors 
include Bayesian methods, the rest of the textbook is covered. Topics include discrete 
probability and combinatorics; random variables; distribution and density functions; moment 
generating functions and moments; sampling theory and limit theorems; estimation and 
hypothesis testing; maximum likelihood and method of moments estimation; efficiency, 
unbiasedness, and asymptotic distribution of estimators; Neyman-Pearson Lemma; goodness- 
of-fit tests; correlation and regression; experimental design and analysis of variance; and 
nonparametric methods. There were multiple instructors who taught these courses for these 
22 capstone students. Data are not available about the students’ performances in the courses, 
though a grade of C or better in both courses was required for registration in the capstone 
course. 

Student work was subjected to multiple rounds of coding (Coffey & Atkinson, 1996). 
Initial rounds of coding focused on correctness and evidence of statistical/distributional 
thinking. Subsequent rounds of coding respected emergent themes and patterns such as the 
use of language. A different coding rubric was used for each the two questions. Details about 
coding strategies are embedded in the Results section of this paper. 

Results 


Figure 1. Question 1. 

Question 1. USA Today published an article called Is 'failure to launch' really a 
failure? (Jayson, 2006). Here are two sentences from it: 

i. “High housing costs are only part of the reason young adults are staying home 
in greater numbers than ever before.” 

ii. “Since 1970, the percentage of people ages 18 to 34 who live at home with 
their family increased 48%, from 12.5 million to 18.6 million, the Census 
Bureau says.” 

(A) Does the second sentence support the statement that “young adults are 
staying home in greater numbers than ever before”? (Assume “young adult” 
means “people ages 18 to 34”.) (B) Is there anything misleading? (C) What 
questions would you like answered in order to further clarify the provided 
statistics? 


Question 1 

Question 1 (see Figure 1) was asked on a homework assignment. Students were not required 
to read the whole article, which used the second sentence in support of the claim that “young 
adults are staying home in greater numbers than ever before”. There were two main issues 
which students were to identify: (1) the article incorrectly stated that the percentage (not the 
number) of people living at home increased, and (2) the article was misleading in its failure to 
acknowledge that, from 1970 to 2006, the population of young adults may have also 
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increased. In fact, the total population increased by approximately 48% (US Census Bureau, 

2012 ). 

Only 19 of the 22 subjects responded to this question; it is unclear why the problem was 
skipped by three students, two of whom completed the other three questions on the 
homework assignment. Of these 19 students, only one made note of Issue (1). In order to 
examine student recognition of Issue (2), responses were initially coded based upon whether 
an issue of proportionality was acknowledged; that is, a student needed to have 
acknowledged base population, either total or for young adults, in order to have demonstrated 
proportional reasoning. Table 1 details codes used to analyze student work. 

Twelve responses clearly addressed proportionality. However, three of these twelve did 
not mention it as the primary reason why the second sentence did not support the phrase 
quoted in part (A); instead they brought up the possibility of a population increase as one of 
their questions in part (C). For example, Student #1 incorrectly focused on the entirety of the 
first sentence from USA today. He wrote, “The second sentence does not say that high 
housing costs are the only reason for the 48% increase.” Among the seven students who did 
not raise the issue of a changing population size, six said that the statistics did not support the 
statement. Two of these students correctly noted that not enough data was provided to justify 
the use of the phrase “than ever before.” Two of the seven proposed a hypothetical situation 
which could have undermined the data; for example, Student #18 wrote that young adults 
“could have moved out and moved back in later.” Two others dismissed the statistic because 
it did not provide a reason for the increase. 


Table 1. Coding for Issue 2 


Code 

Description 

n 

Proportional 
Reasoning (PR) 

Student gave some acknowledgement that increase in 
population (of young adults) is relevant. Student reported 
that quote does not support claim. 

12 

PR: Primary Reason 
(PR:1) 

The primary reason why the quote does not support the 
claim is related to potential increase in population. 

9 

PR: Not Primary 
Reason (PR:2) 

The student response was coded as PR but it was not 
his/her primary reason for concluding that the statistic did 
not support the claim. 

3 

No Proportional 
Reasoning (NPR) 

The student did not raise any issues about potential 
population increase. 

7 

NPR: Supports 
(NPR:S) 

The student response was coded as NPR and the student 
said the statistic does support the claim. 

6 

NPR: Does Not 
Support (NPR:N) 

The student response was coded as NPR and the student 
said the statistic does not support the claim. 

1 

Primary Reason 
(PrimR) 

The primary reason why the quote does not support the 
claim. Code values: population increase, sampling issues, 
hypotheticals, other. 

NA 


Question 2 

Question 2 (see Figure 2) was asked in the context of a take home exam and was answered by 
all 22 students. The intention was for students to engage in distributional thinking. In 
particular, for the students to note that, if the immigrant children’s scores had a similar 
distribution to that of the population of native-born children, then one would expect about 
85% of them to score below the 85th percentile of the total population of students. Responses 
were coded as providing correct reasoning if this reason was identified, even if not as the 
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primary reason. (Table 1 details codes used to analyze student work.) Nine responses were 
coded as correct; however, some mentioned the faulty distributional reasoning only as a 
secondary or tertiary issue. For example, Student #16 (coded as correct), as her primary 
reason, applied the temporal meaning of the word “lagged” and concluded that one could 
only arrive at that conclusion if data were collected over many years. Whereas, Student #1 
was coded as incorrect even though he correctly noted that approximately 25% of the 
immigrant children would have scored above the 85th percentile. However, he may have 
interpreted the article to have implied that 100% of immigrant students lacked proficiency as 
he concluded by asking, “So how is it that 25% of students are considered to be in the group 
of poor performance?” Thus, some students displayed some distributional thinking but did 
not answer the question correctly. 


Figure 2. Question 2. 

Question 2. Here's a quote from the New York Times (Future, 2009). 
“Immigrant children lagged in mastering standard academic English, the 
passport to college and to brighter futures. Whereas native-born children's 
language skills follow a bell curve, immigrants' children were crowded in the 
lower ranks: More than three-quarters of the sample scored below the 85th 
percentile in English proficiency.” 

Does the statement that “more than three-quarters of the sample scored below 
the 85th percentile in English proficiency” support the statement that 
“Immigrant children lagged in mastering standard academic English”? Explain 
why or why not. 


Indeed, the statistic provided in this article contradicts the claim. Only nine of the 22 
students supplied an explanation which acknowledged that a sub-population with only 75% 
scoring below the 85th percentile would likely be outperforming the general population. Of 
the 13 other students, six did demonstrate distributional thinking to some extent. Often, 
however, this thinking was unproductive as was the case with Student #2 who noted that 
“some immigrant children could have scored higher than native bom children." Four other 
students argued that, without knowing how the 75% was distributed, we can’t determine if 
the claim is supported. For example, Student #12 raised the possibility that, “If they fell 
between 80-85 percentile then that would say that three-quarters of [immigrant] children [are] 
80 to 85 more proficient than the whole population which is good.” As can be seen in the 
quote from Student #12, and as was the case with many students, the wording lacked some 
precision. Particularly, it is unclear what Student #12 meant by “80 to 85 more proficient.” 

At least eight of the incorrect answers raised potential sampling or methodological issues 
in ways which did not serve to address the question. For example, Student #17 noted that the 
article did not mention sample size. Student #9 noted that “this article does not specify what 
type of children comprised of this sample which scored below the 85th percentile.” Student 
#8 noted that “the problem with this claim is there is not a cut-off for adequate English 
proficiency” and the results would look different if they focused on the number that achieved 
minimal proficiency. Student #7 suggested that the article should have focused on student 
growth, rather than proficiency. Others, like Student #21, raised questions like “How do 
[they] measure whether or not someone is proficient in English?” As was also the case with 
Question 1, many of these and other points raised by students are valid, though they did little 
to address the question which was asked of them. 
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Code _ 

Distributional Thinking 
(DT) 


Correct/Incorrect 
(C or I) 


Reasons (not necessarily 
primary) 


Table 2. Coding Rubric for Question 2 
Description 

The student demonstrates some reasoning about 
the distributions of the test scores of the sub¬ 
population within the total population. Code 
Values: Yes, No 

Correct if students noted something equivalent 
to the statement, “25% of immigrant children 
would be in the top 15%” and used this to draw 
a correct conclusion. 

The reasons which students gave in support of 
their answers. Only reasons given multiple times 
were assigned codes. Some responses had 
multiple reasons. Code values: correct (C), 
distribution of subpopulation (DS), 
sampling/methodological (SM) _ 


n 

15 (yes), 
7 (no) 


9 (correct), 
13 (incorrect) 


9(C), 

5 (DS), 
8 (SM) 


Limitations 

The implications of the results enumerated above are limited by both the scale and 
methods of the study. The convenience sample of 22 subjects enrolled in a single course at a 
single university clearly limits the generalizability. Furthermore, the study focused on 
students’ work on just two tasks and the validity of these tasks as research instruments was 
not established, for instance, through a panel of experts. The discussion which follows 
respects these limitations and locates the value of this study not in its generalizability, but in 
its potential for advancing an important and timely discussion about the statistical preparation 
of teachers. Limitations are discussed in more depth below, in the context of future research 
which extends this work. 


Discussion 

The formal upper-division courses typically required of mathematics majors may do little 
to prepare them for their careers as mathematics or statistics teachers (CBMS, 2012; Monk, 
1994; Rossman, Chance, Medina, & Obispo, 2006). In the present study, many pre-service 
mathematics teachers, at the end of their undergraduate educations, did not successfully 
evaluate data-based claims. That is, when asked to demonstrate statistical thinking, as defined 
by Chance (2002), by questioning conclusions drawn from data, many fell short. 

Furthermore, though it is difficult to quantify, the PSMTs often struggled when 
communicating about statistics. Rossman et al. (2006) claimed that “communication skills 
and statistical judgment” (p. 332) are largely missing from teacher preparation programs; the 
present study perhaps shows that there are consequences to that lack of instruction. Indeed, 
many of the PSMTs in this study faltered in both communication skills and statistical 
judgment. Makar and Confrey (2005b) made the observation that teachers often used non¬ 
standard terminology to support valid statistical arguments; however, many of the subjects in 
this study used non-standard language while providing incorrect answers. Though, some did 
demonstrate elements of statistical thinking in spite of answering the questions incorrectly. 

The disappointing performance of many of these PSMTs highlights the need for reform in 
the statistical training of secondary teachers that so many have called for. All of the students 
had completed a two-semester calculus-based probability and statistics sequence as the only 
statistics requirement of their degree program. However, on Question 1, only 12 out of 19 
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addressed issues related to proportional reasoning. On Question 2, only nine out of 23 
successfully used distributional thinking. Many of the PSMTs, at the end of their 
undergraduate mathematics education, were unsuccessful with the two statistical tasks which 
echo CCSSM requirement for high school students to “critically [review] uses of statistics in 
public media and other reports” (National Governors Association Center for Best Practices & 
Council of Chief State School Officers, 2010, p. 79). 

There are some notable characteristics of the incorrect answers provided in this study. 
Many incorrect answers were actually valid observations which did not relate to the question. 
This may be an instance of what Kahneman and Frederick (2002) referred to as attribute 
substitution, often characterized by situations “in which a difficult question is answered by 
substituting an answer to an easier one” (p. 50). In particular, students often focused on 
hypothetical methodological issues, which are typically not discussed in news media, but may 
be discussed in statistics courses. That is, the students may have been more comfortable with 
or more predisposed to discussing sampling issues than with a context-based critical 
evaluation of the use of statistics in a non-academic source. The present study does not 
provide insight into the source of this pattern, though it is probable that it is, at least in part, a 
by-product of students’ prior academic experiences with statistics. 

Though, the question remains of what those academic experiences should be for future 
teachers of statistics. The students in this study had taken a Probability and Statistics course 
sequence intended for all mathematics majors. At a moment when K-12 statistics education in 
the United States is aligning with the GAISE and CCSSM visions of school statistics, it is an 
opportune time to question the assumption that a traditional upper division probability and 
statistics sequence develops teachers’ knowledge for teaching statistics. 

As Groth (2007) stated, the task of designing effective programs will require a “daunting 
amount of work” (p. 433). Though, the idea of redesigning the curriculum for PSMTs has 
received traction at least at one university; Froelich, Klieman, and Thompson (2008) describe 
efforts at Iowa State University to redesign the way secondary teachers are trained to teach 
statistics. They have replaced traditional calculus-based statistics courses with content and 
instruction more aligned with the GAISE College Report (2005), incorporating more data 
analysis and conceptual thinking. 

There is a recognized need to better understand the statistical knowledge required for 
teaching (Callingham & Watson, 2011; Groth, 2007). Concomitantly, there is a need to 
examine how teachers are trained, and the assumptions which underlie those teacher training 
programs. More research is needed about the type of statistical training which can support 
teachers of statistics. One avenue toward that goal is to further investigate the ways in which 
traditional statistics coursework may be insufficient. This study examined PSMTs who had 
all completed such courses, however many did not successfully demonstrate some basic 
knowledge needed for statistics teachers. Given the limitations of this study, more research is 
needed to document the extent and nature of the shortcomings which PSMTs may have at the 
end of their mathematics and statistics training. In particular, the present study would be 
strengthened by examining a broader base of PSMTs, by a closer examination of the PSMTs’ 
statistical backgrounds, and by use of a validated, finer-grained research instrument. 
Furthermore, task-based interviews could illuminate the patterns which emerged in the 
students’ errors. 

Such work would support a better understanding of how to address the statistical needs of 
PSMTs. The programmatic changes at Iowa State University are encouraging, though many 
universities are, no doubt, restricted in making such bold changes by departmental politics 
and state-mandated teacher licensing requirements. Research into interventions to supplement 


8 




Issues in the Undergraduate Mathematics Preparation of School Teachers 

ISSN 2165-7874 


existing coursework would create more options for the departments responsible for the 
statistical training of PSMTs. One potential venue for such interventions is in a capstone 
course for mathematics majors. Indeed, the present study documented some capstone 
students’ struggles, yet there is opportunity to measure students’ growth in statistical thinking 
as a consequence of coursework. 

Though the present study has the limitations of a small sample from one institution, it 
adds texture to issues of statistics teacher preparation at a time when most of the United 
States is transitioning to common standards which emphasize interpretation, justification, and 
decision-making in statistics education. If traditional undergraduate coursework is not 
preparing PSMTs to engage in the sorts of statistical thinking that will be required of their 
future students, then it is reasonable to conclude that PSMTs need explicit training in that 
type of statistical thinking. 
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